Assessing accuracy on the front lines: a pragmatic approach for single-donor proficiency testing.

Proficiency testing (PT), also known as external quality assessment (EQA), has earned a well-deserved position as one of the most important elements of a laboratory’s quality management program (1 ). By assessing its performance on blinded samples, a laboratory can determine to what extent its results agree with those of its peers or, ideally, with the reference method. Unfortunately, as has been emphasized recently (1 ), the most frequently used PT materials are noncommutable, meaning that they do not behave exactly like real patient samples. Conclusions drawn from such materials are, of necessity, limited to understanding how well any given peer group performs rather than knowing whether the results on patient samples are, in fact, accurate (2 ). In the case of a method lacking a true peer group (including, for example, the overwhelming majority of mass spectrometry methods and other laboratory developed tests), even this limited type of comparison may not be possible. Additionally, when there are differences between peer groups comprising laboratories that use homogeneous systems (single manufacturer’s instruments, calibrators, and reagents), the differences are often ascribed to lack of commutability (matrix effects) of the materials, although the differences may in fact be real and reflected in patient samples as well (3, 4 ). So-called accuracy-based surveys, which use commutable materials and reference method target values, are becoming more widespread and have generated several improvements (5 ) and interesting findings (6, 7 ). These surveys face some unique challenges. It is difficult to obtain sufficient materials to supply the number of laboratories participating in conventional surveys. It is also difficult to achieve the full range of concentrations one would like; manipulating commutable materials may render them noncommutable (for example, spiking analyte into them to increase concentrations or diluting them to decrease concentrations). In addition, it can be extremely expensive to obtain reference method target values. In this issue of Clinical Chemistry, Stepman et al. (8 ) meaningfully extend a novel approach some of them had previously reported (9 ). They collected 20 single-donor sera according to CLSI protocol 37-A, from which they made 1-mL aliquots, which were frozen at 70 °C until analysis. The authors arranged to have the aliquots analyzed singly for 8 common analytes by 63 individual laboratories. These laboratories were selected to have roughly 10 of each of 6 manufacturers’ homogeneous systems represented. On these samples for these analytes, one would expect agreement to be good, if not excellent. The results, however, indicate that there is much room for improvement. Importantly, because of the study’s design, the authors were able to detect individual sample problems and determine, among other things, individual laboratory imprecision vs that of its peer group and peer group agreement with target values. The authors fully recognize some of the limitations of their study. First, use of the CLSI 37-A protocol to prepare samples did not in itself prove that the samples were commutable. Second, the number of laboratories included was, of necessity, relatively small. Third, the concentration ranges covered by these samples were small, leaving open the question of performance at other concentrations seen in real patient samples. Fourth, for some of the target values, the authors used the “all method trimmed mean” rather than the reference method. Fifth, the authors’ decisions on specific quality limits for individual analytes could be debated. Nonetheless, the insights their data provide are striking. If we cannot achieve agreement on analytes as simple and straightforward as these, is it reasonable to expect to achieve agreement on others (or on these at other concentrations)? Some readers may argue that systematic differences between methods can be overcome by providing different reference intervals. There are several problems with this argument. For one thing, from a strictly scientific perspective, for the 8 analytes discussed in this paper as well as for many others, there are true values, which are the values we should report (10 ). At least as important, though, is that, in many cases (e.g., hemoglobin A1c (5 ) and cholesterol (11 )), physicians use decision limits promulgated by national or inter1 Beth Israel Deaconess Medical Center, Boston, MA. * Address correspondence to this author at: Beth Israel Deaconess Medical Center, Yamins 309, 330 Brookline Ave, Boston, MA 02215. E-mail ghorowit@ bidmc.harvard.edu. Received March 28, 2014; accepted March 31, 2014. Previously published online at DOI: 10.1373/clinchem.2014.224154 2 Nonstandard abbreviations: PT, proficiency testing; EQA, external quality assessment; MDRD, Modification of Diet in Renal Disease. Clinical Chemistry 60:6 806–808 (2014) Editorials